Lecture 18

class: center, middle, inverse, title-slide

# Lecture 18
## Simple Linear Regression
### Psych 10 C
### University of California, Irvine
### 05/11/2022

---

## Simple linear regression

- Last class we mentioned that we can use a line in order to make predictions
about the values of a continuous dependent variable when we have information
about an independent variable.

- We also mentioned that the equation of the line is defined by two parameters:

- `$\beta_0$` which is known as the **intercept** and it can be interpreted as 
  the expected value of the dependent variable when the independent variable
  is equal to 0.
  
--

- `$\beta_1$` which is known as the **slope** which is the change in the 
  expectation of our dependent variable for a **unit** increase in the value
  of our independent variable.
  
--

- The problem was that each combination of values of `$\beta_0$` (**intercept**) 
and `$\beta_1$` (**slope**) will give us a different line, so we need a way to
choose the best one.

---

## Least Squares

- The method that we use to find the values of `$\beta_0$` and `$\beta_1$` is known
as Least Squares. The idea is that we want to choose the values of the 
parameters that minimize the error of the predictions in the model.

- In other words, we want to find the values that minimize the Sum of Squared 
Errors (SSE).

.pull-left[
<img src="data:image/png;base64,#lec-18_files/figure-html/miss-grade-lm1-1.png" style="display: block; margin: auto;" />
]

.pull-right[
<img src="data:image/png;base64,#lec-18_files/figure-html/miss-grade-lm2-1.png" style="display: block; margin: auto;" />
]

---

## Simple linear regression

- Before we find the values of the parameters for the simple linear regression
model, we want to formalize the models that we are going to compare.

- Given that we have a single independent variable `$x_i$`, we will only need to 
compare two models, one that assumes that the independent variable has no 
effect on the values of the dependent variable, and the other one that assumes 
that assumes that the independent variable has an effect.

- Comparing those two models will let us decide if our independent variable is a 
**good predictor** of the dependent variable.

- Before we introduce the models that we want to compare in a simple linear 
regression let's expand the example with grades and missed classes.

---

### Example: grades and classes missed

- We want to know how the grade that a student gets (dependent variable) changes 
as a function of the number of classes they missed during the quarter 
(independent variable).

- In other words, we want to know if the number of classes missed by a student
is a good predictor of their final grade.

- We have the final grade and number of classes missed by 130 students in a 
statistics class:

<div id="htmlwidget-9b83802e2ea71f35187d" style="width:100%;height:auto;" class="datatables html-widget"></div>
<script type="application/json" data-for="htmlwidget-9b83802e2ea71f35187d">{"x":{"filter":"none","vertical":false,"fillContainer":false,"data":[["1","2","3","4","5","6","7","8","9","10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31","32","33","34","35","36","37","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63","64","65","66","67","68","69","70","71","72","73","74","75","76","77","78","79","80","81","82","83","84","85","86","87","88","89","90","91","92","93","94","95","96","97","98","99","100","101","102","103","104","105","106","107","108","109","110","111","112","113","114","115","116","117","118","119","120","121","122","123","124","125","126","127","128","129","130"],[88.5,77,78,74.5,78.5,79,88.5,84.5,75.5,89.5,84,76,76.5,98,84.5,83,60.5,70.5,100,67,71.5,86.5,87,66.5,75.5,69.5,87.5,72.5,89.5,100,77.5,76,92.5,79.5,65,65.5,71.5,100,70,85.5,94,78.5,81,91.5,78.5,85,90.5,97.5,73.5,82.5,84,95,91.5,90.5,76,73.5,86,71.5,90.5,72,68,77,73,83.5,77.5,90,67,75,89.5,77,59.5,82.5,75.5,75,80,93.5,71,88,85.5,85,87.5,83.5,72.5,50.5,85,68,100,74,87,89,61.5,67,82.5,84,83,63,94,87.5,82,73.5,87,91,94.5,89,81.5,87.5,83.5,90,83,55.5,80.5,76.5,86,65,74.5,80,75,89.5,97,70.5,85,91,72,98,89,71,92.5,98,64.5,86],[3,4,6,7,9,4,3,3,3,5,2,4,5,4,1,10,7,5,2,6,9,5,6,7,7,7,3,7,5,3,1,6,5,3,4,5,9,5,2,9,4,3,2,1,5,4,3,5,3,5,4,4,3,5,6,1,2,3,7,6,4,6,4,7,7,10,6,6,5,4,7,7,1,12,6,3,6,6,3,4,5,3,7,7,4,8,5,6,6,2,11,8,5,8,6,6,4,7,6,3,4,2,5,4,5,3,5,2,6,7,3,3,6,6,5,4,8,1,4,7,4,6,6,2,6,10,3,2,7,2]],"container":"<table class=\"display\">\n  <thead>\n    <tr>\n      <th> <\/th>\n      <th>grade<\/th>\n      <th>classes_missed<\/th>\n    <\/tr>\n  <\/thead>\n<\/table>","options":{"pageLength":4,"columnDefs":[{"className":"dt-right","targets":[1,2]},{"orderable":false,"targets":0}],"order":[],"autoWidth":false,"orderClasses":false,"lengthMenu":[4,10,25,50,100]}},"evals":[],"jsHooks":[]}</script>

---

## Models for simple linear regression

- The first model that we need to compare the Null model.

- The Null model in a simple linear regression formalizes the assumption that 
the expected value of our dependent variable is constant regardless of the 
values of the predictor (independent variable).

- This model is expressed formally as: `$$y_i \sim \text{Normal}(\beta_0, \sigma_0^2)$$`

- Were `$i$` denotes the observation number.

- In our example, this model assumes that the expected grade of a student is 
independent of the number of classes they missed.

- Notice that the only difference between this Null model and ones that we have 
seen before is that we denote the expectation as `$\beta_0$` instead of `$\mu$`.

- Given that we only have one value to minimize the error of our predictions our
best guess for the value of `$\beta_0$` for the Null model will be the group 
average.

---

## Simple linear regression model

- The second model that we need to evaluate is the linear regression.

- This model assumes that the expected value of our dependent variable is a 
linear function of the number of classes missed by a student.

- Formally, the model is expressed as: 
`$$y_i \sim \text{Normal}(\ \beta_0+\beta_1x_i,\sigma_1^2)$$`

- In other words, the model assumes that all observations with with the same 
value of the predictor `$x_i$` follow the same distribution, but this distribution 
is not the same for every value of `$x$`.

- In our grades example this means that the model assumes that the expected
grade of students that missed 3 classes is `$\beta_0 +\beta_1\ 3$` and that
distribution is different from students that missed 5, 6 or any other number of
classes.

- Notice that the variance is the same regardless of the number of classes that 
the student missed.

---

## Simple linear regression

- Another way to think about this is that the linear model predicts a different
expected grade for each number of classes missed.

- This would be similar to the multiple groups case that we talked about on week 
4.

- However, a simple linear regression has the advantage that it assumes that 
the change in the expected value of the dependent variable should follow a 
straight line.

- Now we can look at the values of the parameters for each model.

---

## Predictions: Null model

- As we said before, the Null model assumes that the expected grade of a student 
is constant (doesn't change) as a function of the number of classes they missed.

- Given that the model has only one parameter, our best guess for the value of 
that parameter will be the average of the dependent variable of all participants. 
This is the same estimator that we have used for the Null model before.

```r
n_total <- nrow(grades)

null <- grades %>% 
  summarise("pred" = mean(grade))

grades <- grades %>% 
  mutate("prediction_null" = null$pred,
         "error_null" = (grade - prediction_null)^2)

sse_null <- sum(grades$error_null)
mse_null <- 1/n_total * sse_null
bic_null <- n_total * log(mse_null) + 1 * log(n_total)
```

---

## Simple linear regression

- Now we need to get the values for `$\hat{\beta}_0$` and `$\hat{\beta}_1$` for the 
simple linear regression.

- There is no function that I know in `tidyverse` that will allow us to do this,
however, it is easy to do in base R.

- The R function to do a linear regression in base R is `lm()`.

- There are 2 important arguments of this function that we need to use. The 
first one is `formula =` and it requires us to express the model as:
`$$\text{dependent-variable} \sim \text{independent-variable}$$`

- The second argument for the function is the `data =` where we have to indicate
the name of the object that contains our observations.

---

## Parameters: Simple linear regression

- First we will get the values of `$\hat{\beta}_0$` and `$\hat{\beta}_1$`.

```r
betas <- lm(formula = grade ~ classes_missed, data = grades)$coef
```

- The function `lm()` will return multiple values so to get only the values of 
`$\hat{\beta}_0$` and `$\hat{\beta}_1$` we need to use `$coef` after the function.

- Now we can add the predictions and errors of the linear model to our data, 
remember that the prediction of the linear model will be:
`$$\hat{\beta}_0 + \hat{\beta}_1 \text{classes-missed}_i$$`

```r
grades <- grades %>% 
  mutate("prediction_linear" = betas[1] + betas[2] * classes_missed,
         "error_linear" = (grade - prediction_linear)^2)
```

---

## Model comparison

- As with the previous problems, we can get the SSE, Mean SE, `$R^2$` and BIC for
the models in a linear regression.

```r
sse_linear <- sum(grades$error_linear)
mse_linear <- 1/n_total * sse_linear
r2_linear <- (sse_null - sse_linear)/sse_null
bic_linear <- n_total * log(mse_linear) + 2 * log(n_total)
```

- We can compare the models using a table:

| Model | Parameters | MSE | `$R^2$` | BIC |
|-------|:----------:|:---------------------:|:-----:|:--------------------:|
| Null  | 1          | 103 | NA    | 608|
| Classes Missed  | 2          | 87 | 0.16 | 591|

- This means that including the classes missed by a student as a predictor of
final grade improves the model in comparison to using the mean.

---

## Interpretation

- The value of `$R^2$` is interpreted the same way as before, it is the proportion
of error accounted for by the model in comparison to the Null model.

- Now we can interpret the values of the coefficients `$\hat{\beta}_0$` and 
`$\hat{\beta}_1$`.

- Students that missed 0 classes have on average a final grade of approximately
  90 `$(\hat{\beta}_0)$`.
  
--

- According to the model, the number of points that a student is expected to 
  lose for every class missed is approximately -2 
  `$(\hat{\beta}_1)$`.
  
--

- Using `ggplot` we can visualize the predictions of a simple linear regression.

- We call this type of data visualization a `scatterplot`.

---

count: false
 
### Scater plot grade by classes missed
.panel1-scatter-code-auto[

```r
*ggplot(data = grades)
```
]
 
.panel2-scatter-code-auto[
![](data:image/png;base64,#lec-18_files/figure-html/scatter-code_auto_01_output-1.png)
]